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Intellectual Property Rights 



IPRs essential or potentially essential to the present document may have been declared to ETSI. The information 
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found 
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in 
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web 
server ( http://webapp.etsi.org/IPR/home.asp) . 

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee 
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web 
server) which are, or may be, or may become, essential to the present document. 



Foreword 

This Technical Specification (TS) has been produced by ETSI Technical Conmiittee Speech and multimedia 
Transmission Quality (STQ). 

The present document is part 1 of a multi-part deliverable covering the QoS aspects for popular services in mobile 
networks, as identified below: 

Part 1 : "Assessment of Quality of Service" ; 

Part 2: "Definition of Quality of Service parameters and their computation" ; 

Part 3: "Typical procedures for Quality of Service measurement equipment" ; 

Part 4: "Requirements for Quality of Service measurement equipment" ; 

Part 5 : "Definition of typical measurement profiles" ; 

Part 6: "Post processing and statistical methods"; 

Part 7: "Network based Quality of Service measurements" . 

The present document builds an umbrella document for this multi-part deliverable. It summarizes the basics of Quality 
of Service, always seen from the user's perspective. Differences to Quality of Experience (QoE) are also discussed. In 
extension to generic definitions, specific definitions for this multi-part deliverable are stated here. Furthermore, it gives 
guidance to assure that QoS assessments can be conducted in a meaningful way and proposes an according process. 

Part 2 defines QoS parameters and their computation for popular services in mobile networks. The parameter definition 
is split into several parts. It contains an abstract definition which gives a generic description of the parameter, an 
abstract equation and the corresponding user and technical trigger points. 

The harmonized definitions given in part 2 are considered as prerequisites for the comparison of QoS measurements and 
measurement results. 

Part 3 describes the measurement procedures needed to perform the measurements of QoS parameters in line with the 
definitions given in part 2, applying the test profiles defined in part 5. 

Part 4 defines the minimum requirements of QoS measurement equipment for mobile networks in the way that the 
values and trigger points needed to compute the QoS parameter as defined in part 2 can be measured following the 
procedures defined in part 3. Test equipment fulfilling the specified minimum requirements will allow performing the 
proposed measurements in a reliable and reproducible way. 

Part 5 specifies typical measurement profiles which are required to enable benchmarking of different mobile networks 
both within and outside national boundaries. 

Part 6 describes procedures to be used for statistical calculations in the field of QoS measurement of mobile networks 
using probing systems. 

Part 7 describes how Quality of Service measurements should be done inside the network without direct access to the 
end point terminal. 
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Introduction 



The term Quality of Service (QoS) is extensively used today, not just in the telecommunication world in which it has its 
roots, but increasingly regarding broadband, wireless and multimedia services that are based on the IP protocol. 
Networks and systems are gradually being designed in consideration of the end-to-end performance required by user 
applications; however, the term QoS is usually not well-defined, is used loosely or, worst of all, misused. Therefore, 
guidance is needed on how to understand and apply the term QoS. 

The term "Quality of Service" addresses technical as well as non-technical aspects affecting a service. Different 
concepts and guidance have been developed to cover various interests and viewpoints of all parties of 
telecommunications service market, i.e. users, service providers, network operators, manufacturers and regulators. 

In many cases, the user and his or her expectations with respect to quality are in the centre of interest. This very generic 
topic can have manifold characteristics: 

• Depending on the role of the user, his expectations may vary: Considering service quality he might have other 
expectations compared with situations that are oriented more commercially, e.g. when he is in contractual 
discussions with his service provider. 

• Not only service usage is relevant to the user. The overall impression of all touch points with his provider is 
influencing his personal quality reception. 

• Furthermore, the user compares his expectations with the reached level of fulfilment. Future decisions will be 
based on his personal perception of the achieved level of quality. In this case, subjective components get an 
increased weight and importance. Taking these aspects also into account, the term "Quality of Service" has to 
be extended to "Quality of Experience" with a more subjective meaning. 

This multi-part deliverable covers all facets which are relevant to the assessment of Quality of Service as seen from a 
user's perspective but with a technical interpretation. Whereas the "user's perspective" reflects events and triggers 
observable by a user, the "technical interpretation" is related to the fact that quality statements should be reproducible, 
comparable and reliable. Often automation techniques are used to achieve these goals based on a statistically valid data 
basis. 

From a more practical view, different assessment methods are discussed. In a further step, the preparation and execution 
of assessment procedures are shown as well as the generation of key performance indicators, their aggregation and their 
matching against pre-defined target values. Finally, consecutive steps like optimization procedures are concluding this 
process-like view of QoS. 

Also belonging to the more practical clause of the present document, basic definitions of measures are provided. To 
generate a common understanding of service independent and service dependent measures is also a goal which the last 
clauses of the present document should achieve. 

In detail, the present document describes Quality of Service from a more theoretical and a more practical view. 

Clauses 4 and 5 build the theoretical background of all Quality of Service related matters: 

• Clause 4 reflects all relevant definitions and abbreviations which are used in terms of Quality of Service. They 
are compiled from different sources like reference documents, involved services and standardization 
discussions. 

• Clause 5 contains background information to Quality of Service. This rather theoretical clause discusses 
generic definitions of Quality of Service, Performance and Quality of Experience. Furthermore, QoS models 
defined in further standard documents and their interrelations are a subject of discussion. 

Clauses 6 to 9 lead step by step from theoretical discussions on QoS to issues which are of practical and pragmatically 
relevance: 

• Clause 6 presents a sequential order which describes a QoS assessment as being a process. Starting with the 
definition of targets and required preparation steps, the execution of the assessment as well as validation, 
reporting and optimization matters are discussed. 

• Clause 7 provides basic definitions which are required to have a common understanding on fundamental 
topics. Examples are question like "What is a kilobyte?" or "How to define a timeout value?". 
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• Clause 8 deals with service independent QoS criteria, namely QoS parameters which are of relevance before a 
service is used. This covers e.g. the connection dialup via mobile networks. 

• Clause 9 handles service dependent QoS criteria. Depending on the service, different parameters are required 
or not to give a complete picture of this service's QoS. 

The standardization work in the QoS area is still ongoing. Therefore, the definitions and procedures given in will have 
to be reviewed on a regular basis to keep them up to date. The information contained in the present document will form 
among other input the basis for further work, but is likely to be modified and amended. Therefore, it is recommended to 
cross-check the given information with actual discussions within ITU-T and ETSI and with standards published after 
the date of publication of the present document. 
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Scope 



The present document serves as a generic umbrella document for the further documents part 2 to part 7 of this series. It 
gives an overview over the topics addressed by these documents and enables the reader to work with the documents in 
the intended way. It is important to understand that the complete series of documents focuses on Quality of Service 
which stands for the objective discussion of quality measures from a rather technical perspective. Based on existing 
quality standards and further definitions, a complete picture of Quality of Service as seen from a user's point of view is 
drawn. 

Wherever possible, existing ITU-T or ETSI definitions are referenced. If ITU-T or ETSI definitions do not exist or are 
considered as too generic, a more service and mobile network specific definition is made. 

The present document comprises the theoretical backgrounds to understand terms like "Quality of Service", "Network 
Performance" and "Quality of Experience". Their meaning and interrelation are discussed by taking different QoS 
models into account. 



References 



References are either specific (identified by date of publication and/or edition number or version number) or 
non-specific. For specific references, only the cited version applies. For non-specific references, the latest version of the 
reference document (including any amendments) applies. 

Referenced documents which are not found to be publicly available in the expected location might be found at 
http ://docbox . etsi . or g/Ref erence . 

NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee 
their long term validity. 

2.1 Normative references 

The following referenced documents are necessary for the application of the present document. 

[1] ITU-T Recommendation E.800: "Definitions of terms related to quality of service". 

[2] ITU-T Recommendation G.IOOO: "Communications Quality of Service: A framework and 

definitions". 

[3] ETSI TS 102 250-7:" Speech and multimedia Transmission Quality (STQ); QoS aspects for 

popular services in GSM and 3G networks; Part 7: Network based Quality of Service 
measurements". 

2.2 Informative references 

The following referenced documents are not necessary for the application of the present document but they assist the 
user with regard to a particular subject area. 

[i.l] ITU-T Recommendation 1.350: "General aspects of quality of service and network performance in 

digital networks, including ISDNs". 

[i.2] ITU-T Recommendation P.IO/G.IOO: "Vocabulary for Performance and QoS". 

[i.3] ITU-R Recommendation BT.500-1 1 : "Methodologie d'evaluation subjective de la qualite des 

images de television". 

[i.4] ITU-T Recommendation X.745: "Information technology - Open Systems Interconnection - 

Systems Management: Test management function". 

[i.5] ITU-T Recommendation G.109: "Definition of categories of speech transmission quality". 
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[i.6] ITU-R Recommendation P. 800: "Methods for subjective determination of transmission quality". 

[i.7] ITU-T Recommendation E.802: "Framework and methodologies for the determination and 

application of QoS parameters " . 

[i.8] ETSI TR 102 493 (Vl.2.1): " Speech and multimedia Transmission Quality (STQ); Guidelines for 

the use of Video Quality Algorithms for Mobile Applications". 



3 Abbreviations 

For the purposes of the present document, the following abbreviations apply: 

FEC Forward Error Correction 

FTP File Transfer Protocol 

IP Internet Protocol 

MAC Medium Access Control 

MO Mobile Originating 

MOS Mean Opinion Score 

NP Network Performance 

PCO Point of Control and Observation 

POR Point Of Recording 

PTN Network Termination Point (PTN) 

QoE Quality of Experience 

QoS Quality of Service 

QoSD Quality of Service Delivered 

QoSE Quality of Service Experienced 

QoSO Quality of Service Offered 

QoSR Quality of Service Required 

RLC Radio Link Control 

RNC Radio Network Controller 

RRC Radio Resource Control 

SLA Service Level Agreement 

SQL Structured Query Language 

UNI User Network Interface 



4 Guidance on commonly used Terms and Definitions 

Throughout this multi-part deliverable, the following terms and definitions apply: 

NOTE 1: Several terms and definitions presented in this clause are defined in ITU-T Recommendation E.800 [1], 
ITU-T Recommendation E.802 [i.7] and G.IOOO [2]. 

• 1 kByte: 1 024 Byte. 

• 1 MByte: 1 024 kByte. 

• Access Point Name: Is used to identify a specific IP network and a point of interconnection to that network. 

• Active Testing: Refers to the way that data is acquired actively for the measurement, i.e. that the test makes 
use of a dedicated channel for the measurement, e.g. by dialling a number and making a call, i.e. setting-up a 
channel for the measurement. 

• A-party: Initiating part of a connection (also: Mobile Originating, MO) OR in direct transactions, the party 
initiating the transaction (calling party). 

NOTE 2: In store-and-forward transactions, the party sending content. 

• Benchmark: Evaluation of performance value/s of a parameter or set of parameters for the purpose of 
establishing value/s as the norm against which future performance achievements may be compared or 
assessed. 
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• B -Party: In direct transactions, the termination or counterpart of a transaction. 
NOTE 3: In store-and- forward transactions, the party receiving content. 

Broadcast: Information transfer from one transmitting entity to many receiving entities. 

Content: Entirety of information transferred within a transaction, seen from the user's perspective. 

NOTE 4: In case of services requiring entrance procedures (e.g. server login with FTP), information flow to 
achieve the state of being able to transfer actual user data is not counted as content. 

Cut-off: Unintended termination of a communication session. 

Data Service: Telecommunications service involving the transport of data via the PTN such that any user can 
use equipment connected to a network termination point to exchange data with another user of equipment 
connected to another termination point. 

Direct Service: Service which makes use of direct communications between a client entity and a server entity 
without persistent storage of transferred data in interconnected network elements. 

Download: Transfer of data or programs from a server or host computer to one's own computer or device. 

Drive test tool: end-point test tool which is designed to be moved around, i.e. by walking or driving a car. 

Email: Messages automatically passed from one computer user to another, often through computer networks 
and/or via modems over telephone lines. 

End-to-end Quality: Quality related to the performance of a communication system, including all terminal 
equipment. For voice services it is equivalent to mouth-to-ear quality. 

Event: In this multi-part deliverable, an event is understood as a change of condition (the according point of 
time is considered in addition). 

Host: An entity that provides client stations with access to files and printers as shared resources to a computer 
network. 

Idle Mode: A communication device is in this state when it is powered-on but not transmitting a signal 
Intrusive Testing: According to the definitions ITU-T Recommendation X.745 [i.4], clause 3.10.3, intrusive 
test means: "A statement made with respect to a test invocation if service/user disruption will or may occur as 
a result of the test". This refers to the way that data is acquired for the measurement, i.e. whether or not 
sending a specific predefined and known reference signal over a channel for analysis purposes is required. 

NOTE 5: In contrast to active testing, intrusive testing means that a test signal is sent over the network. 

NOTE 6: The combinations "active and intrusive testing" and "passive and non-intrusive testing" define the most 
common test situations. 

IP Service Access: Basic access to the generic packet-data transfer capabilities the service is based upon. 

Landing Page: The first website that appears in the Internet browser when a user tries to browse the Internet. It 
is often used to allow the user to make some specific settings for the following Internet session. 

Maximum Expected Delivery Time: For store-and-forward services, this defines the time span within which a 
message shall be received by the B -party to rate the transaction successful from the user's perspective. 

Mean Data Rate: Average data rate of a data transmission, calculated by dividing the number of transmitted 
bits by the duration of the transmission. 

Mean Value: In this multi-part deliverable, the mean value is understood as the estimated expectation value of 
a distribution. See also arithmetic mean definition from statistics or part 6 of this series. 

Network Access: Access to the network under test. 

Network Accessibility: The probability that the user of a service after a request (to a network) receives the 
proceed-to-select signal within specified conditions. 
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• Network Availability: Probability of success of network functions performed by a network over a specified 
time interval. 

• Network Operator: Organization that provides a network for the provision of a public telecommunication 
service. 

• Non-intrusive Testing: According to the definitions in clause 3.10.5, non-intrusive test means: "A statement 
made with respect to a test invocation if no service/user disruption will or may occur as a result of the test" . 
This refers to the way that data is acquired for the measurement, i.e. whether or not sending a specific 
predefined and known reference signal over a channel for analysis purposes is required. 

• Passive Testing: Refers to the way that data is acquired passively for the measurement, i.e. that the test makes 
use of an existing channel for the measurement, e.g. by tapping a further defined point of this channel. 

• Probing Attempt: Trial to examine if the service under test works as expected. 

• QoS Criterion: Single characteristic of a product or service that is observable and/or measurable. 

• QoS Indicator: A characteristic that is used to determine the Quality of Service. 

• Quality: The totality of characteristics of an entity that bear on its ability to satisfy stated and implied needs. 

NOTE 7: The characteristics should be observable and/or measurable. When the characteristics are defined they 
become parameters and are expressed by metrics. 

• Rate: Change of amount of a quantity divided by the portion of time during which it has been changed. 
NOTE 8: The denominator's unit is related to time. 

• Ratio: measurement result which represents a subgroup of all single measurements is related to the total 
number of executed single measurements. 

NOTE 9: Usually, nominator and denominator share the same unit, namely a counter for measurements 
(subgroup/all). 

Reliability: The probability that an item can perform a required function under stated conditions for a given 
time interval. 

Retrieval: Transport of content from network to B -party, initiated by the B -party. 

Service: A set of functions offered to a user by an organisation constitutes a service. 

Service Access: A set of functions offered to a user by an organisation constitutes a service. 

Service Integrity: The degree to which a service is provided without excessive impairments, once obtained. 

Service Retainability: Service retainability describes the termination of services (in accordance with or against 
the will of the user). 

Session: Continuous usage of a given service, e.g. a speech call or a data session. 

Session Time: Duration of a session. 

Setup: The period starting when the address information required for setting up a call is received by the 
network (recognized on the calling user's access line) and finishing when the called party busy tone, or ringing 
tone or answer signal is received by the calling party (i.e. recognized on the calling user's access line). Local, 
national and service calls should be included, but calls to Other Licensed Operators should not, as a given 
operator cannot control the QoS delivered by another network. 

Speech Quality: Quality of spoken language as perceived when acoustically displayed. Result of a perception 
and assessment process, in which the assessing subject establishes a relationship between the perceived 
characteristics, i.e. the auditory event, and the desired or expected characteristics. 
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Speech Transmission Quality: Speech quaUty related to the performance of a communication system, in 
general terms. Categories of speech transmission quality are defined in ITU-T Recommendation G.109 [i.5], 
based on the prediction of the E- model, i.e. in terms of ranges for the transmission rating factor R. 

Store and Forward: Store-and-forward services are services where content is stored in the network and 
delivered to the recipient to a later point in time. 

Streaming: Multimedia data (usually combinations of voice, text, video and audio) transferred in a stream of 
packets that are interpreted and rendered, by a software application as the packets arrive. 

Talk Burst: Flow of media, e.g. some seconds of speech, from a terminal while that has the permission to send 
media. 

Test case: Consists of a number of single identical transactions. 

Timeout: Specified period of time that will be allowed to elapse in a system (e.g. inactivity) before a specified 
event is to take place. 

Transaction: Single, complete, typical usage of a particular service. 

Trigger: Always defined with respect to a particular transaction - An event adopting one of the trigger roles for 
a particular transaction. 

Trigger Event: See "Trigger". 

Trigger Point: Absolute time (a.k.a. "point in time") of occurrence of a trigger event. Note, however, that this 
term is widely used in a less restrictive manner, meaning either the event or its time of occurrence, 
respectively, depending on the context. 

A specific event adopting one of the trigger roles for a particular transaction is often referred to as "technical 
trigger point" (for this transaction). It occurs / is measured at a specific PCO. 

User Equipment: Technical device in user's possession, used for communication purposes. 

Video: A signal that contains timing/synchronization information as well as luminance (intensity) and 
chrominance (colour) information that when displayed on an appropriate device gives a visual representation 
of the original image sequence. 



QoS Background 



ITU-T Recommendation E.800 [1] provides the basic definition of QoS and highlights operational aspects of providing 
networks and services. In doing so, ITU-T Recommendation E.800 [1] already gives a QoS definition and a framework 
for QoS implementation. 

The definition given in ITU-T Reconmiendation E.800 [1] is as follows: 

Totality of characteristics of a telecommunications service that bear on its ability 

to satisfy stated and implied needs of the user of the service. 

Thus, in general QoS is focused on the service from the user's viewpoint being a complete end-to-end view. However, 
since the QoS consists of the collective effect of numerous single performances, any QoS analysis will have to deal also 
with sub-parts, e.g. network and terminal performance that can be analyzed separately and independently from another. 
Therefore, there are a lot of standards and concepts dealing with QoS that are focusing on specific details and aspects of 
QoS. 
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The perceived quality of a service is the result of the combined influence of the performance of networks and terminals 
as well as the perception and expectation of the user. Thus QoS should also take into account both the user's and the 
service provider's point of view; it should always be judged from these different perspectives. There is an interrelation 
between user's requirements and his perception of the delivered quality on the one hand and the service/QoS planned 
and achieved by the service provider on the other hand. 

A comprehensive view on QoS should take into account all aspects and perspectives including the numerous standards 
dealing with specific sub-parts of QoS. In the following clauses basic issues that need to be considered are discussed in 
more detail. 



5.1 



End-to-End QoS 



As already indicated QoS covers the whole end-to-end view of a telecommunications service and can be subdivided in 
separate parts that all have an influence on the resulting QoS. The degree of QoS depends on the collective effect of all 
sub-parts. This is illustrated in Figure 5.1. 

The Figure 5.1 is inspired by ITU-T Recommendation E.800 [1]. 
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Figure 5.1 : End-to-end QoS 

Quality measures in telecommunications can be determined in a hierarchical manner: 

• Network Performance (NP): The network performance is assessed across a part of a network or a 
sub-network. Mostly, the NP is given in a technical way by assessing technical parameters which describe the 
performance of this part of the network in the desired way. Examples are parameters like bit error ratio, 
sending and receiving power, transmission delay, etc. 

• Overall NP: If several network sections should be considered as being one integral part of the network 
("black box"), the overall network performance has to be assessed. For example, the network performance of 
the complete network transmission between the two User Network Interfaces (UNI) can be summarized in this 
way. 

• End-to-end Quality of Service (QoS): The assessment of the overall transmission chain from a user's 
perspective is considered to deliver the Quality of Service in an objective manner. This implies that the most 
complete transmission chain without involving the user himself should be taken into account. Mostly, the 
generated measures rely on service related characteristics without knowing any details about the underlying 
network sections which are required to have an end-to-end service at all. 
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• Quality of Experience (QoE): The inclusion of the user himself to the overall quality in telecommunications 
extends the rather objective Quality of Service to the highly subjective Quality of Experience. The QoE differs 
from user to user since it is influenced by personal experiences and expectations of the individual user. 

5.2 Relationship of QoS and Performance 

It is important to understand that QoS differs from network and terminal performance. QoS is the outcome of the user's 
experience/perception, while the network and terminal performance is determined by the performances of network 
elements one-by-one, or by the performance of the network as a whole including the performance of the attached 
terminals, i.e. the combination of the performance of all single elements. This means that the network performance may 
be used with an end-to-end meaning, but it may also be used to describe the performance of a network section. 

EXAMPLE: Access performance is usually separated from the core network performance in the operations of a 
single IP network, while Internet performance often reflects the combined NP of several 
autonomous networks. 

However, the network and terminal performance have an influence on the QoS; they represent a part of it. The 
combined effect of the performance of all elements determines the overall service performance. There are intrinsic 
relationships between QoS and performance parameters, the former having a direct or indirect, and sometimes even 
inverse, influence on the latter. Furthermore, some performance measures can have a direct QoS meaning, while some 
others have to be combined in order to have a QoS signification. 

Performance parameters are used to measure objectively the performance of specific network and terminal elements that 
have an influence on the resulting end-to-end quality of a service. Performance is measured and expressed in 
performance parameters. The main difference between QoS and network performance is that QoS provides quality 
information on an end-to-end and service related basis, whereas network performance specifies the technical 
operativeness of network and terminal elements or of network sections. 

ITU-T Recommendation 1.350 [i.l] provides the following conceptual categorization of Quality of Service (QoS) and 
Network Performance (NP) metrics as follows: 



Quality of Service parameter 


Networl< Performance parameter 


User oriented 


Network provider oriented 


Service related attributes 


Network element and technology related attributes 


Focus on user observable effects 


Focus on planning development (design), operations and 
maintenance 


Observed at service access points for the users, 
independent of network process and events 


Observed at network connection element boundaries, e.g. 
relating to protocol specific interface signals 



5.3 Relationship of QoS and QoE 



In addition to the term QoS, the term Quality of Experience (QoE) is often used nowadays in order to stress the purely 
subjective nature of quality assessments in telecommunications and its focus on the user's perspective of the overall 
value of the service provided. 

The increased significance of the term QoE is related to the fact that in the past the term QoS was used laxly and mostly 
for only technical concepts focused on networks and networks elements. The definition of QoS, however, does include 
the degree of satisfaction of a user with a service. Thus, non-technical aspects are included, like e.g. the user's 
environment, his expectations, the nature of the content and its importance. But most service providers did use the QoS 
only in relation to the actual user-service interaction in order to cross-check whether the user requirements have been 
met by the service implementation of a provider (as perceived by the user). So there was a strong focus on the actual 
network performance and its immediate influence on user perceivable aspects while additional subjective and not 
directly service related aspects were omitted. 

QoE is defined in ITU-T Recommendation P.IO/G.IOO [i.2] in Appendix I as the overall acceptability of an application 
or service, as perceived subjectively by the end-user. It includes the complete end-to-end system effects (client, 
terminal, network, services infrastructure, etc) and may be influenced by user expectations and context. Hence the QoE 
is measured subjectively by the end-user and may differ from one user to the other. However, it is often estimated using 
objective measurements. 
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Contributing to the QoE are objective service performance measures such as information loss and delay. Those 
objective measures together with human components that may include emotions, linguistic background, attitude, 
motivation, etc. determine the overall acceptability of the service by the end-user. Figure 5.2 shows factors contributing 
to QoE. These factors are organized as those related to Quality of Service and those that can be classified as human 
components. 

QoE for video is often measured via carefully controlled subjective tests (ITU-R Recommendation BT.500-11 [i.3] and 
ITU-R Recommendation P. 800 [i.6]) where video samples are played to viewers, who are asked to rate them on a scale. 
The rating assigned to each case are averaged together to yield the mean opinion score (MOS). 

Quality of service (QoS) is defined in ITU-T Recommendation E.800 [1] as the collective effect of performance which 
determines the degree of satisfaction of a user of the service. In general, QoS is measured in an objective way. 

In telecommunications, QoS is usually a measure of performance of services delivered by networks QoS mechanisms 
include any mechanism that contributes to improvement of the overall performance of the system and hence to 
improving the end-user experience. QoS mechanisms can be implemented at different levels. 

EXAMPLE: At the network level, QoS mechanisms include traffic management mechanisms such as buffering 
and scheduling employed to differentiate between traffic belonging to different applications. Other 
QoS mechanisms at levels other than the transport include loss concealment, application Forward 
Error Correction (EEC), etc. 

QoS parameters are used to describe the observed QoS. Similar to the QoS mechanisms, QoS parameters can be defined 
at different layers. Figure 5.2 gives an impression on factors that have an influence on the QoS and the QoE. 



Objective 



QoE 



Quality of 
Semce 



Subjective 



Human Components 



Service 
factors 



Tlanspoit 
factors 



Application 

factoi'S 



Emotioiis 



Setvice billing 



Experience 



Figure 5.2: QoE Dimensions 

In general, there is a correlation between the subjective QoE as measured by the MOS and various objective parameters 
of Quality of Service. 

Typically, there will be multiple service level performance (QoS) metrics that impact overall QoE. The relation between 
QoE and service performance (QoS) metrics is typically derived empirically. Having identified the QoE/QoS 
relationship, it can be used in two ways: 

1) Given a QoS measurement, one could predict the expected QoE for a user. 

2) Given a target QoE for a user, one could deduce the net required service layer performance. 

These prediction and deduction steps are built on assumptions and approximations. Due to the complexity of services 
and the many factors which have an influence on QoS/QoE, there is not a close one-to-one relationship which would 
allow statements like "If the bandwidth is increased by 200 kbit/s, the rating by the user will rise 0,5 points". 

To ensure that the appropriate service quality is delivered, QoE targets should be established for each service and be 
included early on in system design and engineering processes where they are translated into objective service level 
performance metrics. 
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Quality of Experience will be an important factor in the marketplace success of triple-play services and is expected to 
be a key differentiator with respect to competing service offerings. Subscribers to network services do not care how 
service quality is achieved. What matters to them is how well a service meets their expectations for effectiveness, 
operability, availability, and ease of use. 



5.4 QoS Models in Standardisation Documents 

The relationship between user satisfaction, QoS and Network Performance is shown in Figure 5.3. The present 
document has its focus on the technical aspects related to user satisfaction. 
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Figure 5.3: Relationship between user satisfaction, QoS and Network Performance 

5.4.1 Model of ITU-T Recommendation G.1000 

In ITU-T Recommendation G.IOOO [2], a four- viewpoint model of QoS has been defined. 
In general, the model discusses two dimensions: 

• The relationship between a service user, the "customer", and his service provider; and 

• The QoS expectations and the achieved level of QoS for both parties, the customer and the service provider. 
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Figure 5.4 describes the different Quality of Service relations is inspired by ITU-T Recommendation G.IOOO [2]. 
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Figure 5.4: Four-viewpoint model of QoS 

The expected and perceived QoS from the user's perspective might be given in more descriptive terms whereas the 
provider will use more technically oriented terms to handle his offered and achieved level of QoS. 



5.4.1.1 



QoS Required by the User (QoSR) 



The QoS requirements of the user are described in a common, non-technical language. To describe his needs, the user 
reflects his expectations from an end user's perspective. This means he formulates the requirements he expects from the 
services delivered over the network. The user needs not to be aware of the technical feasibilities or implementation 
limitations that may occur. 

Depending on the boundary conditions, the required QoS might also be part of contractual terms. 

The user's QoS requirements are the basis for the QoS level which should be offered by his service provider. The 
service provider should take the given requirements to deliver a QoS level which is matching the user's needs. 

5.4.1 .2 QoS Offered by the Service Provider (QoSO) 

The service provider states the QoS level he wants to reach. This can be done in two ways: 

• In a non-technical manner to ease the comprehensibility of the given information towards the user. 

• In a technical manner to allow an assessment by according experts, to allow to set up Service Level 
Agreements (SLA) or to ease technical planning purposes. 

For technical purposes, the QoS level is defined by the use of parameter definitions and according values which should 
be reached. This information should be given separately for each kind of offered service. 



5.4.1.3 



QoS Delivered by the Service Provider (QoSD) 



The QoS delivered by the service provider reflects the currently achieved state of the QoS. This QoS level again should 
be described by parameters with assigned values, e.g. from active or passive probing or other kinds of appropriate 
testing. 

The comparison of the offered and the delivered QoS allows assessing the capabilities of the service provider to deliver 
the promised QoS. Deviations can be made transparent very easily. 
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5.4.1.4 



QoS Experienced by the User (QoSE) 



The experienced or perceived QoS reflects the subjective view of the user on his individual situation. The user's 
satisfaction is one of the main drivers of this kind of QoS. 

In general, the perceived QoS is described in a non-technical manner. Service providers can retrieve the level of 
perceived QoS by executing surveys with their customers or by asking their users for other kind of feedback. 

At this stage, the user combines his personal and individual experience with the more technically oriented quality of the 
delivered services. Overall, his individual Quality of Experience measure is generated. In technical means, QoE and 
QoS can be mapped on each other. 

Besides the technology based factors, further factors have an influence on the QoE of a user: Starting with the signature 
of the contract, going on with handling of problems by the provider, willingness to fulfil the user's needs and other 
things up to the cessation of the contract, the complete relationship between provider and user might have an influence 
on the QoE. 

Obviously, the relationship experience has an influence on rating issues and makes the mapping of QoS and QoE more 
complicated because of "hidden factors". 

5.4.2 Model of ITU-T Recommendation E.800 

The ITU-T Recommendation E.800 [1] illustrates the relationship between QoS and Network Performance by 
discussing the overall transmission chain which is used to deliver services to the user. 

Figure 5.5 shows a scenario where two mobile users are communicating with each other. In general, the same situation 
applies also to any other constellation where a user deploys a client- server-like service. 
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Figure 5.5: Transmission chain discussed in ITU-T Recommendation E.800 [1] 

In the depicted case, there are two user equipments and two access networks, one on each end. The core network builds 
the link between both access networks. It may consist of different networks run by different providers. 
Each of the mentioned components has an influence on the achievable QoS level: 

• If one of the user equipments has limited capabilities, e.g. reduced computational power, this will have an 
observable effect on the end-to-end QoS. 

• The same applies to the access networks, where e.g. the bandwidth of the transmission link has a major effect. 

• Furthermore, if one of the providers linked within the core network violates SLAs between the providers, the 
end user may realise this by the QoS he perceives. 



ETSI 



19 



ETSI TS 102 250-1 V2.2.1 (2011-04) 



Figure 5.6 gives a more abstract definition of QoS. 
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Figure 5.6: Building blocks for Quality of Service 

The QoS perceived by a user is on the one hand influenced by technical terms Hke accessibihty of a service or the setup 
delay for dialup connections. On the other hand, factors like tariffs, repair times, hotline reachability and many others 
build the non-network performance. 

Both components are integral parts of the end-to-end QoS perceived by the user. 

5.4.3 Phase oriented Aspect Model 

The usage of a service can also be separated in different consecutive phase. Here, the order in time is the criterion which 
differentiates the states of the service usage. 

Figure 5.7 shows the different phases of network access, service access and service usage and the according QoS 
aspects. 
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Figure 5.7: QoS aspects related to different phases of service usage 

The meaning of these phase related QoS aspects (Figure 5.7) is: 

1) Network Availability: Probability that the services are offered to a user via a network infrastructure. 

2) Network Accessibility: Probability that the user performs a successful registration on the network which 
delivers the service. The network can only be accessed if it is available to the user. 

3) Service Accessibility: Probability that the user can access the service he wants to use. A given Network 
Accessibility is a precondition for this phase. 

4) Service Integrity: This describes the Quality of Service during service use and contains elements like the 
quality of the transmitted content, e.g. speech quality, video quality or number of bit errors in a transmitted 
file. The Service Integrity can only be determined if the service has been accessed successfully. 

5) Service Retainability: Service retainability describes the termination of services (in accordance with or 
against the will of the user). Examples for this are all kinds of cut-off parameters, e.g. the call cut-off ratio or 
the data cut-off ratio. Again, a previously performed successful service access is a precondition for this phase. 
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It is important to understand the interaction between these phases. As mentioned above, the phases depend on each 
other. Only if the previous phase has been passed successfully, the parameters of the consecutive phase can be 
determined. 

Success of a phase is defined by successful execution of an attempt and in-time reaction of the network. 

In most cases, this is the occurrence of a certain event (referenced as the "stop trigger") within a predefined time period 
(referenced as "timeout period"). 

The phase is not concluded successfully if at least one of the mentioned components is missing: 

If another than the specified event occurs, e.g. an event representing an error constellation. 

If the specified event occurs, but after expiration of the timeout period. 

If no event occurs at all. 

While some of the phases have a greater importance for mobile networks, the general concept can be applied to any 
kind of network delivering services to the user. 



6 QoS Assessment Process 

To get reliable, reproducible and plausible QoS results, it is recommended to follow a generic scheme. This scheme is 
known as a QoS assessment process. 

Following this predefined scheme, all relevant clauses are covered to reach the mentioned aim. 

6.1 Objective of a QoS Assessment 

The objective of the criteria list is to have an agreed set of QoS criteria. They should allow easier external and internal 
benchmarking. 

The services chosen are considered to be of a high relevance to the user in a national and international market and are 
common for most of the network operators. 

The selected criteria are considered: 

• to have main influence on the user's satisfaction with regard to the service; 

• to identify technical QoS aspects, which can be influenced by the performance of the network or the terminal; 

• to be measurable by technical means; 

• to be relevant for network operator's national and international benchmarking. 

There is the need to specify independent QoS criteria for each service; whereas related preconditions may be necessary. 

6.2 Boundary conditions for a QoS Assessment 
6.2.1 Setting the Target of the Assessment 

This clause deals with areas which have to be considered when defining the target of an assessment. 

• One-time snapshot 

• Acceptance procedure 

• Continuous monitoring 

• Optimisation cycle 
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Benchmarking 



6.2.2 Defining the Boundary Conditions 



Furthermore, lots of different boundary conditions have been defined unambiguously to allow comparable, reliable and 
reproducible results. In general terms, the basic questions "What, when, which way, what not, where, who" have to be 
answered in advance to the execution of the assessment. 

• Set of QoS parameters to be assessed. 

• Excluded topics. 

• Number of samples to be generated per QoS parameter to achieve a pre-defined uncertainty level (determines 
the test duration in the end). 

• Definition of parameters like file sizes, timeout conditions, bearers to use. 

• Foreseen timeframe: Overall timeframe, operating hours per day. 

• Foreseen locations: Included areas, excluded areas. 

• Modes of testing: Active/passive, intrusive/non-intrusive. 

• Mode of automation: Manual testing, automated testing, autonomous testing. 

• Mobility modes: Static, nomadic, drive testing. 

• Used test platform: Host computer, mobile device, observation platform within the network. 



6.2.3 Operational Issues 



In this clause, operational issues to execute the assessment are addressed. These considerations are still to be taken into 
account before the assessment starts. 

Important questions to be answered are: 

How will the assessment be executed? 

Which personnel are involved? Which qualifications are required? 

Which manning is required? (e.g. one engineer, two campaign managers, four drive testers) 

Which stand-ins are available? 

How is operational security assured? 

Which backup procedures are applied? 

6.3 Execution of a QoS Assessment 

Practical steps from defining dedicated boundary conditions (place, time, duration, services, devices, network 
technologies, etc.) to handling of retrieved raw data before validation (backup, inspection of sampling during test, etc.) 
are considered at this stage: 

Is a validation step required? How will it be done? Which documentation is required? 

Which steps are to be done for data generation? 

How is data stored? 

Is there an online reporting? 

Is there an online monitoring? 
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• What is the protection of the data transmission chain (e.g. injection of SQL queries, cross side scripting, etc.)? 

6.4 Validating and Aggregating Results of a QoS Assessment 

When the data collection phase of the assessment is finalized, the gathered data has to be checked in different ways 
before aggregations and further QoS parameter calculations are made: 

Checking the amount of generated data. 

Execution of plausibility checks. 

Removal of erroneous samples (e.g. due to system problems) by setting according markers. 

Generated data should be marked as invalid instead of being deleted. 

Aggregation of data according to areas of interest (time, location, provider, QoS parameter). 

Execution of plausibility checks on aggregated data to detect possible systematic errors (e.g. deviation of the 
mean value of a QoS parameter while the observed error rate is in the normal range). 

Calculation of minimum and maximum values. 

Quantile values, typically 5 % and 95 % quantiles which give some outlier related information. 

Compressed footprints of collected data by giving a set of quantile values, e.g. 5 %, 10 %, 50 % (median), 
90 % and 95 % quantiles, extended by the mean value. 

6.5 Reporting Results of a QoS Assessment 

After the evaluation steps are done in detail, a set of QoS parameter is available. These QoS parameters are 
characteristic for the dedicated assessment and represent the outcome of the overall activity. They should be reported in 
an easy understandable format and should point out the key findings for each relevant constellation, e.g. for each 
question and also for each provider. 

Lots of different schemes can be applied to visualise the determined QoS parameter sets. Since in many cases the 
readers of such reports are used to have a certain representation, changes in the reporting format should be applied 
carefully. 

Finally, the report including some analysis and visual representations should be distributed to the stakeholders. 

6.6 Matching QoS Results with Targets 

Based on the retrieved QoS results, the current state of QoS parameters should be matched against the according 
predefined target values. By doing this matching, deviations from the desired QoS state get obvious. Possible results 
might be: 

• The current QoS situation is better than expected. QoS targets are reached completely. In this situation, one 
can think of defining tight targets or of relaxing QoS requirements to save operational costs. This is the most 
convenient situation. 

• The current QoS situation matches exactly the target level. In this case, the QoS process delivers a stable 
outcome. For the future, the trend of the achieved QoS level has to be observed to assure that QoS targets are 
reached in long-term. 

• The current QoS situation does not reach the expected QoS level. In this case, some actions should be taken to 
improve the QoS level. It is not recommended to just relax the required QoS level to reach the QoS targets 
again! 
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6.7 Optimization of QoS Matters 



If a negative deviation of the QoS level has been stated, some correcting actions should take place. In the end, the 
desired QoS level should be reached again. 

Optimization efforts are always a matter of technical and economical feasibility. This means there must be a chance to 
change the technical implementation of a service and there must be available financial and personnel resources to 
implements the foreseen changes. If one of these factors fails, an effective optimization and therefore improvement of 
the QoS level is not achievable. 



Basic Settings for QoS Assessments 



This clause defines generic statements which should ease the readers' efforts to implement and execute according QoS 
assessments. The main aim is to assure reproducible, reliable and comparable results of QoS assessments. 

7.1 Location where the Measurement is actually performed 
7.1.1 Concept of PCOs 

The Point of Control and Observation (from now on called "point of observation" or PCO) is the location where the 
measurement is actually performed. The location can be either inside the network or in the end-point. The 
measurements should be done using standardized interfaces and protocols. 

Possible points of observation for QoS parameters covered in the present document are: 

• Inside nodes in the network (RNC, base station, switch, etc.) 

• Observations in the terminal: 

End-point test tool; or 

Measurements that are reported back from the terminal to the network 
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(can be multiple PCO) 



PCO in network node 
or at network node 
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Figure 7.1 : Visualisation of different PCOs 
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7.1 .2 Point of Recording (POR) 



The point of recording (POR) is where the QoS parameters are recorded. The POR can be the same as the PCO or 
another point inside the terminal or the network. If the PCO and the POR are not the same, the measurement data must 
be reported from the PCO to the POR. Examples of such reporting are described in annex A of TS 102 250-7 [3] 

7.2 Usage of Standardized Units for Data 

In this multi-part deliverable, the following definitions related to the amount of data have to be applied: 

• 1 kByte is defined as the amount of 1 024 Byte. 

• 1 MByte is defined as the amount of 1 024 kByte which is equivalent to 1 048 576 Bytes. 

This multi-part deliverable orientates itself on units like kByte or MByte to describe the amount of data, e.g. in a storage 
or during a data transfer. 

Therefore, units like MibiBytes (MiB) which are officially standardised in lEC/IEEE will not be applied here. 

7.3 Influence of Timeout Values on Failure Ratios 

For many QoS parameters, especially those from the service phase "Accessibility", there are pairs of QoS parameters 
available. Mostly, there are two QoS parameters defined: 

• one QoS parameter describes the success or non- success of a setup trial (ratio QoS parameter); and 

• a second QoS parameter describes the setup delay for those attempts where the setup has been successful 
(delay QoS parameter). 

Following the principles of this multi-part deliverable, both QoS parameters should be based on the same trigger points 
wherever possible. 

However, both QoS parameters are not decoupled in general. The link between them is the timeout value which is 
always required to determine the delay QoS parameter in a defined manner. The timeout value assures that the system 
under test waits only for a predefined period of time before the attempt is terminated. 

By choosing long intervals for the timeout period, two effects occur: 

1) The system waits for a long period of time before closing down the attempts. This will reduce the number of 
samples per hour for example. 

2) By waiting for a long period of time, the probability is high that the successful event (here the successful 
connection establishment event) will be observed. This will increase the first QoS parameter describing the 
setup success ratio. 

On the other hand, short intervals for the timeout period have these effects on the QoS parameters: 

1) The system waits for a short period of time before closing down the attempts. This will increase the number of 
samples per hour for example. 

2) By waiting for a short period of time, the probability is high that the timeout duration is reached often. Once 
the timeout condition is reached, the ratio QoS parameter counts an unsuccessful attempt. The according delay 
QoS parameter cannot be determined in this case. 

Comparing both scenarios, the overall influence of the choice of the timeout value for the outcome of both QoS 
parameter values should be obvious. 

In simple words: the timeout value influences heavily the sample number per hour as well as the observed setup success 
ratio. The shorter the timeout value is chosen, the lower the setup success ratio will be, and the less delay QoS 
parameter values will be available. 
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8 Service Independent QoS Criteria 

This clause gives some hints how some of the very basic service independent terms are used in this 
multi-part deliverable. 

8.1 Unavailability 

Unavailability describes the probability that a network or service is not offered to a user. 



8.2 Non-Accessibility 



Non-accessibility parameters handle the probability that the user cannot perform a successful access to a network or 
service when he intends to use the network or service. 

8.3 Time Parameters 

Different time parameters occur in QoS measurements. Typical representatives are time parameters like Access Time, 
Activation Time or Setup Time. 

8.4 Transfer Time 

The transfer time is a basic parameter to calculate data rates, meaning to divide the amount of transmitted data by the 
time period which has been required to transfer this amount of data. 



8.5 Content Integrity 



Content integrity parameters describe the quality of the transferred data. Typical representatives are parameters like 
"Speech Quality", "Video Quality" or "Data Integrity". Related to video quality, appropriate guidelines to determine the 
quality of video content in the right manner can be found in TR 102 493 [i.8]. 



9 Service dependent QoS Criteria 

This clause gives some hints how some of the very basic service dependent terms are used in this multi-part deliverable. 

9.1 Rate Parameters 

After a data link has been successfully established, these parameters describe the average data transfer rate measured 
throughout the entire connect time to the service. The data transfer shall be successfully terminated. The prerequisite for 
these parameters is network and service access. 

Rate parameters always have a relation to time. For example, a data rate expresses the relation of an amount of data 
which is transferred within a specific period of time. 

The unit of rate parameters always carries a time unit in the denominator, like in kbit/s. 

9.2 Ratio Parameters 

Ratio parameters reflect the relation between a subset and a basic set. For example, an error ratio reflects the number of 
errors which could be observed in relation to all observed attempts, tries or executions. 

In general, dedicated preconditions have to be fulfilled before an attempt can be added to the basic set of attempts. 
These preconditions depend on the service the ratio parameter should be determined for. 
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The unit of ratio parameters is always a percentage, identified by the % character. A typical representative of a ratio 
parameter is a success ratio, which might reach any value between % (meaning all attempts failed) and 100 % 
(meaning all attempts were executed successfully). 



9.3 Service Non-Accessibility 



The service non-accessibility ratio denotes the probability that a subscriber cannot access the desired service 
successfully. This includes all necessary steps like a required connection establishment. 

In general, dedicated preconditions have to be fulfilled before this parameter can be assessed. E.g. for voice telephony, 
the network must be available, the terminal must be attached to the network and the addressed B -party should not be 
busy. 

9.4 Setup Time 

The setup time describes the time period needed to access the service successfully. 

EXAMPLE: Related to voice telephony, the setup time describes the period of time from starting the dial-up 

connection to the point of time when the content is sent or received. 

This parameter is not calculated unless the according setup attempt is successful. 

9.5 Failure Ratio 

A failure ratio is the probability that a user experiences a malfunction of a certain requested transaction. Typical 
examples are setup failure ratios. 

9.6 Cut-off Ratio 

A cut-off ratio describes the probability that a user experiences an unintended termination of a transaction. Typical 
examples are voice cut-off ratio or streaming cut-off ratio. The precondition for the assessment of this parameter is the 
successful establishment attempt, e.g. for a voice call or a data session. 

9.7 End-to-end Failure Ratio 

The term end-to-end failure ratio defines the probability that a transaction with end-to-end meaning fails. End-to-end 
meaning represents the fact that the whole transmission chain from the data source to the data sink is included in this 
consideration. 

Typically, failures in a part of the end-to-end transmission chain will be reflected also in the end-to-end failure ratio. 



9.8 End-to-end Delivery Time 



The end-to-end delivery time represents the delay that occurs when content is transferred across the complete 
transmission chain from the data source to the data sink. 

One has to differentiate additionally between a unidirectional transmission and a bidirectional transmission where the 
transmission chain is passed twice. 
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